<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.7: http://docutils.sourceforge.net/" />
<title>Software Carpentry Bootcamp Beta Testing</title>
<meta name="authors" content="Matthew McCormick (spidr)" />
<meta name="organization" content="The Hacker Within" />
<style type="text/css">

/*
:Author: David Goodger (goodger@python.org)
:Id: $Id: html4css1.css 6253 2010-03-02 00:24:53Z milde $
:Copyright: This stylesheet has been placed in the public domain.

Default cascading style sheet for the HTML output of Docutils.

See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
customize this style sheet.
*/

/* used to remove borders from tables and images */
.borderless, table.borderless td, table.borderless th {
  border: 0 }

table.borderless td, table.borderless th {
  /* Override padding for "table.docutils td" with "! important".
     The right padding separates the table cells. */
  padding: 0 0.5em 0 0 ! important }

.first {
  /* Override more specific margin styles with "! important". */
  margin-top: 0 ! important }

.last, .with-subtitle {
  margin-bottom: 0 ! important }

.hidden {
  display: none }

a.toc-backref {
  text-decoration: none ;
  color: black }

blockquote.epigraph {
  margin: 2em 5em ; }

dl.docutils dd {
  margin-bottom: 0.5em }

/* Uncomment (and remove this text!) to get bold-faced definition list terms
dl.docutils dt {
  font-weight: bold }
*/

div.abstract {
  margin: 2em 5em }

div.abstract p.topic-title {
  font-weight: bold ;
  text-align: center }

div.admonition, div.attention, div.caution, div.danger, div.error,
div.hint, div.important, div.note, div.tip, div.warning {
  margin: 2em ;
  border: medium outset ;
  padding: 1em }

div.admonition p.admonition-title, div.hint p.admonition-title,
div.important p.admonition-title, div.note p.admonition-title,
div.tip p.admonition-title {
  font-weight: bold ;
  font-family: sans-serif }

div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title {
  color: red ;
  font-weight: bold ;
  font-family: sans-serif }

/* Uncomment (and remove this text!) to get reduced vertical space in
   compound paragraphs.
div.compound .compound-first, div.compound .compound-middle {
  margin-bottom: 0.5em }

div.compound .compound-last, div.compound .compound-middle {
  margin-top: 0.5em }
*/

div.dedication {
  margin: 2em 5em ;
  text-align: center ;
  font-style: italic }

div.dedication p.topic-title {
  font-weight: bold ;
  font-style: normal }

div.figure {
  margin-left: 2em ;
  margin-right: 2em }

div.footer, div.header {
  clear: both;
  font-size: smaller }

div.line-block {
  display: block ;
  margin-top: 1em ;
  margin-bottom: 1em }

div.line-block div.line-block {
  margin-top: 0 ;
  margin-bottom: 0 ;
  margin-left: 1.5em }

div.sidebar {
  margin: 0 0 0.5em 1em ;
  border: medium outset ;
  padding: 1em ;
  background-color: #ffffee ;
  width: 40% ;
  float: right ;
  clear: right }

div.sidebar p.rubric {
  font-family: sans-serif ;
  font-size: medium }

div.system-messages {
  margin: 5em }

div.system-messages h1 {
  color: red }

div.system-message {
  border: medium outset ;
  padding: 1em }

div.system-message p.system-message-title {
  color: red ;
  font-weight: bold }

div.topic {
  margin: 2em }

h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
  margin-top: 0.4em }

h1.title {
  text-align: center }

h2.subtitle {
  text-align: center }

hr.docutils {
  width: 75% }

img.align-left, .figure.align-left, object.align-left {
  clear: left ;
  float: left ;
  margin-right: 1em }

img.align-right, .figure.align-right, object.align-right {
  clear: right ;
  float: right ;
  margin-left: 1em }

img.align-center, .figure.align-center, object.align-center {
  display: block;
  margin-left: auto;
  margin-right: auto;
}

.align-left {
  text-align: left }

.align-center {
  clear: both ;
  text-align: center }

.align-right {
  text-align: right }

/* reset inner alignment in figures */
div.align-right {
  text-align: left }

/* div.align-center * { */
/*   text-align: left } */

ol.simple, ul.simple {
  margin-bottom: 1em }

ol.arabic {
  list-style: decimal }

ol.loweralpha {
  list-style: lower-alpha }

ol.upperalpha {
  list-style: upper-alpha }

ol.lowerroman {
  list-style: lower-roman }

ol.upperroman {
  list-style: upper-roman }

p.attribution {
  text-align: right ;
  margin-left: 50% }

p.caption {
  font-style: italic }

p.credits {
  font-style: italic ;
  font-size: smaller }

p.label {
  white-space: nowrap }

p.rubric {
  font-weight: bold ;
  font-size: larger ;
  color: maroon ;
  text-align: center }

p.sidebar-title {
  font-family: sans-serif ;
  font-weight: bold ;
  font-size: larger }

p.sidebar-subtitle {
  font-family: sans-serif ;
  font-weight: bold }

p.topic-title {
  font-weight: bold }

pre.address {
  margin-bottom: 0 ;
  margin-top: 0 ;
  font: inherit }

pre.literal-block, pre.doctest-block {
  margin-left: 2em ;
  margin-right: 2em }

span.classifier {
  font-family: sans-serif ;
  font-style: oblique }

span.classifier-delimiter {
  font-family: sans-serif ;
  font-weight: bold }

span.interpreted {
  font-family: sans-serif }

span.option {
  white-space: nowrap }

span.pre {
  white-space: pre }

span.problematic {
  color: red }

span.section-subtitle {
  /* font-size relative to parent (h1..h6 element) */
  font-size: 80% }

table.citation {
  border-left: solid 1px gray;
  margin-left: 1px }

table.docinfo {
  margin: 2em 4em }

table.docutils {
  margin-top: 0.5em ;
  margin-bottom: 0.5em }

table.footnote {
  border-left: solid 1px black;
  margin-left: 1px }

table.docutils td, table.docutils th,
table.docinfo td, table.docinfo th {
  padding-left: 0.5em ;
  padding-right: 0.5em ;
  vertical-align: top }

table.docutils th.field-name, table.docinfo th.docinfo-name {
  font-weight: bold ;
  text-align: left ;
  white-space: nowrap ;
  padding-left: 0 }

h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
  font-size: 100% }

ul.auto-toc {
  list-style-type: none }

</style>
</head>
<body>
<div class="document" id="software-carpentry-bootcamp-beta-testing">
<h1 class="title">Software Carpentry Bootcamp Beta Testing</h1>
<h2 class="subtitle" id="build-systems">Build Systems</h2>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Authors:</th>
<td>Matthew McCormick (spidr)</td></tr>
<tr><th class="docinfo-name">Organization:</th>
<td>The Hacker Within</td></tr>
<tr><th class="docinfo-name">Contact:</th>
<td><a class="first last reference external" href="mailto:hacker.within.admin&#64;gmail.com">hacker.within.admin&#64;gmail.com</a></td></tr>
</tbody>
</table>
<div class="section" id="welcome">
<h1>Welcome</h1>
<p>Welcome to Software Carpentry Beta Testing session on automated Documentation.
This training course is being taught by members of The Hacker Within, a
scientific computing interest group based in the Engineering Physics Department
but open to anyone. In short, the purpose of The Hacker Within is to help you be
a more productive and efficient researcher and student---and to put you in
contact with others trying to do likewise.</p>
<p>To sign up for <a class="reference external" href="http://groups.google.com/group/hacker-within}">our Google Group</a> email mailing list, visit it
and click <em>Apply for group membership</em> or
<a class="reference external" href="mailto:hacker.within.admin&#64;gmail.com">email us</a>.</p>
<p>We'll start today's session by introducing the instructor(s) ( Matt McCormick ) and
distributing temporary CAE user names and passwords.</p>
</div>
<div class="section" id="introduction">
<h1>Introduction</h1>
<p>What can a build system do for your project?</p>
<ul class="simple">
<li>Make it <strong>configurable</strong>.</li>
<li>Create a <strong>Make</strong> system that speeds up development.</li>
<li>Make it <strong>installable</strong>.</li>
</ul>
<div class="section" id="configuration-system">
<h2>Configuration system</h2>
<p>Build systems often provide configuration capabilities that address all aspects
of the project that may be variable in the build phase of a project.  This
usually involves an auto-detection system and sensible defaults coupled with
methods to override the defaults and specify values the auto-detection system
failed to find.</p>
<p>Things that can be configured include</p>
<ul class="simple">
<li>Platform<ul>
<li>Operating system<ul>
<li>Linux</li>
<li>Mac</li>
<li>Windows</li>
<li>embedded</li>
</ul>
</li>
<li>Architecture<ul>
<li>Endianness</li>
<li>32/64 bit</li>
</ul>
</li>
</ul>
</li>
<li>Libraries<ul>
<li>Binaries</li>
<li>Header files</li>
<li>Versions</li>
</ul>
</li>
<li>Compiler and other build tools</li>
<li>User preferences<ul>
<li>Compile flags</li>
<li>What optional parts of the project to build<ul>
<li>Support a file format</li>
<li>Build a plugin</li>
</ul>
</li>
<li>Non-standard or different dependency location</li>
<li>Installation location</li>
</ul>
</li>
</ul>
</div>
<div class="section" id="make-system">
<h2>Make system</h2>
<p>What is a Make system?</p>
<ul class="simple">
<li>Manage the multiple steps of the C/C++/fortran build process<ul>
<li>Pre-process</li>
<li>Compile</li>
<li>Link</li>
</ul>
</li>
<li>Pass appropriate flags at each step.</li>
<li>During the development process, <em>only</em> recompile and link files that have
<em>changed</em>.</li>
</ul>
<p>What are examples of Make system?</p>
<ul>
<li><p class="first">Unix <tt class="docutils literal">Makefile</tt> and <tt class="docutils literal">make</tt></p>
</li>
<li><p class="first">Integrated development environment project files.</p>
<p>Sidebar: Why not write your own Makefile?</p>
<p>Writing your own Makefile from scratch can be time consuming and error-prone.
E.g., many make implementations are sensitive to having spaces instead of tabs
in the Makefile.  Also, if the Makefile is generated by build system, the
configuration capabilities of the build system can make the Makefile much more
flexible.</p>
</li>
</ul>
</div>
<div class="section" id="installation-system">
<h2>Installation system</h2>
<p>Want other people to use your project?  You need an installation system!</p>
<p>On Windows, this means creating a <tt class="docutils literal">Setup.exe</tt>.  On Unix systems, this means
generating a Makefile with an <em>install</em> target so that <tt class="docutils literal">make install</tt> can be
called.</p>
<blockquote>
<p>Sidebar: Why not write a simple script to perform the installation?</p>
<p>The user may want to eventually upgrade or even uninstall your program,
fanstastic as it may be.  Due to tradition, the installation program is
usually created by the application developer, but the uninstallation program
is usually the responsibility of the operating system.  On Windows, this is
the 'Add/Remove Programs'.  On Unix, this is the responsibility of the package
manager.  This means the the installation program needs special
platform-dependent capabilities,
which are usually already taken care of by the build system.</p>
<p>For example, on Linux <tt class="docutils literal">make install</tt> is not used when creating packages.
Instead, <tt class="docutils literal">make <span class="pre">DESTDIR=&lt;a_fake_root_dir&gt;</span> install</tt> installs the package to a
fake root directory.  Then, a package is created from the fake root directory,
and uninstallation is possible because a manifest is generated from the
result.</p>
</blockquote>
</div>
</div>
<div class="section" id="c-example-with-cmake">
<h1>C++ example with CMake</h1>
<p><a class="reference external" href="http://cmake.org/">CMake</a> is a C/C++ build system (with some additional support for Fortran, Java, and Python
extensions).  The website description:</p>
<blockquote>
Welcome to CMake, the cross-platform, open-source build system. CMake is a
family of tools designed to build, test and package software. CMake is used to
control the software compilation process using simple platform and compiler
independent configuration files. CMake generates native makefiles and
workspaces that can be used in the compiler environment of your choice.</blockquote>
<div class="section" id="get-the-example-code">
<h2>Get the example code</h2>
<p>Our first order of business is to get you access to these notes and other course
materials. We've placed them in a repository on our <a class="reference external" href="http://code.google.com/p/hacker-within/">Google Code site</a>.
However, instead of having you visit the site and download them manually, we
will download the latest version with version control.</p>
<p>You may check out a read only or a committing copy of the repository. To check
out a read only copy of today's session:</p>
<pre class="literal-block">
svn checkout http://hacker-within.googlecode.com/svn/trunk/sc/documentation/src cxx-example
</pre>
</div>
<div class="section" id="build-the-example">
<h2>Build the example</h2>
<p>CMake is designed to keep files that result from the build process separate from
the source code directory.  This makes it easier to keep the source directory
under version control, and it makes it easy to start a fresh configuration and
build with a simple <tt class="docutils literal">rm <span class="pre">-rf</span></tt> on the build directory.</p>
<p>Let's make a build directory:</p>
<pre class="literal-block">
mkdir cxx-example_build
cd cxx-example_build
</pre>
<p>CMake has a Turing complete scripting language with scripting directives
designed specifically for a C/C++ build system.  These scripts are kept in files
named <tt class="docutils literal">CMakeLists.txt</tt>.</p>
<p>Now lets run the CMake curses gui on our CMake configured source code.  The
executable <tt class="docutils literal">ccmake</tt> takes as an argument the path to the top level
CMakeLists.txt.:</p>
<pre class="literal-block">
ccmake ../cxx-example
</pre>
<p>This simple user interface can be used to change configuration variables.  There
is one important exception to be aware of -- the compiler.  Since changing
compilers in the middle of a build is generally dangerous, this is not allowed.
To specify a different compiler, it must be specified at the time of initial
configuration.  This can be done with traditional environmental variables.  For
example, if we want to use <tt class="docutils literal"><span class="pre">g++-4.3</span></tt>:</p>
<pre class="literal-block">
rm -rf *
CXX=g++-4.3 ccmake ../cxx-example
</pre>
<p>Instructions for using the curses can be found at the bottom of the interface.
Another tip is use <em>/</em> to search through options like in <tt class="docutils literal">less</tt>.  Press <em>c</em> to
do the initial configuration.  The system environment will be polled and the
appropriate settings for the build will be defined.  Depending on the complexity
of the project or to deal with changing options, <em>c</em> may need to be pressed
multiple times.  Finally, press <em>g</em> to generate the Makefile.  Now we list the
contents of the directory:</p>
<pre class="literal-block">
ls
</pre>
<p>And we see that a Makefile has been generated along with some other files.:</p>
<pre class="literal-block">
CMakeCache.txt  CMakeFiles/  cmake_install.cmake  doc/  Makefile
</pre>
<p>We run <tt class="docutils literal">make</tt>, which will use the instructions in the Makefile to build the
object files and link them together into an executable.:</p>
<pre class="literal-block">
make
</pre>
<p>If we look at the contents of the directory again, we see our <tt class="docutils literal">oop</tt> executable
has been built.  Try executing it:</p>
<pre class="literal-block">
./oop
</pre>
<p>We see some neat population dynamics.</p>
</div>
<div class="section" id="using-our-configuration-capabilities">
<h2>Using our configuration capabilities</h2>
<p>Let us use our configuration capabilities by changing some compiler flags.</p>
<p>First, let us do a simple benchmark of our program's execution with the Unix
<tt class="docutils literal">time</tt> command.:</p>
<pre class="literal-block">
time ./oop
</pre>
<p>Make a note of how long it took to execute the program.</p>
<p>We can remove all our built object files, libraries, and executables
with:</p>
<pre class="literal-block">
make clean
</pre>
<p>Now, let us run <tt class="docutils literal">make</tt> again, but have also print out the commands it is
executing while it is running.:</p>
<pre class="literal-block">
make VERBOSE=1
</pre>
<p>Now, let us add some additional compiler flags.  First, let us open the
configuration interface:</p>
<pre class="literal-block">
ccmake .
</pre>
<p>CMake allows one to change through some common build setting profiles with the
<em>CMAKE_BUILD_TYPE</em>.  Change the CMAKE_BUILD_TYPE to <em>RelWithDebInfo</em>.
<em>RelWithDebInfo</em> stand for &quot;Release with debugging information.&quot;  Press <em>t</em> to
toggle advanced mode.  The default compile flags for the RelWithDebInfo
CMAKE_BUILD_TYPE are &quot;-O2 -g&quot;, i.e. do optimization level 2 and include
debugging symbols.  Let use also add optimizations that will be specific for the
current machine's architecture by adding &quot;-march=native&quot; to <em>CMAKE_CXX_FLAGS</em>.
Then <em>c</em> and <em>g</em>.  For more information on what the flags do, see <tt class="docutils literal">man gcc</tt>.</p>
<p>Now we build our files again, but we see our flags are also passed to g++.:</p>
<pre class="literal-block">
make VERBOSE=1
</pre>
<p>Now, let us see if the execution speed of our program has changed with the
optimization flags.:</p>
<pre class="literal-block">
time ./oop
</pre>
</div>
<div class="section" id="what-the-make-system-does-for-us">
<h2>What the Make system does for us</h2>
<p>Again, remove our built files.:</p>
<pre class="literal-block">
make clean
</pre>
<p>Let us time how long it takes to build our project.:</p>
<pre class="literal-block">
time make
</pre>
<p>Now let us time how long it would take to build the project just using g++
directly, which you might have done for smaller projects.:</p>
<pre class="literal-block">
time g++ -O2 -g -march=native ../cxx-example/*.cpp -o ./oop
</pre>
<p>You should get a similar time or slightly faster.</p>
<p>OK, but let us simulate a development situation where you changed a file, saved
it, then need to rebuild.  To simulate a file save let us use <tt class="docutils literal">touch</tt>:</p>
<pre class="literal-block">
touch ../cxx-example/main.cpp
</pre>
<p>The manual method behaves the same:</p>
<pre class="literal-block">
time g++ -O2 -g -march=native ../cxx-example/*.cpp -o ./oop
</pre>
<p>Now let us rebuild with the make system:</p>
<pre class="literal-block">
time make
</pre>
<p>We only had to re-compile the file we changed!  This can make a huge difference for
larger projects.</p>
<p>The make system also allows us to compile in parallel.  Do the following to
count the number of processors on your system:</p>
<pre class="literal-block">
cat /proc/cpuinfo | less
</pre>
<p>Hit the <em>PageDown</em> key to scroll through the output.  Specify the number of
parallel compiles with the <em>-j</em> switch to <tt class="docutils literal">make</tt>.  A good choice is the number
of processors plus two.  So, try:</p>
<pre class="literal-block">
make clean
time make -jN
</pre>
<p>Where <em>N</em> is the number of processors plus two.  You should see a significant
increase in speed (you may not if someone else is using some of the processors
remotely on your CAE machine).</p>
</div>
<div class="section" id="demonstrating-the-installation-capabilities">
<h2>Demonstrating the installation capabilities</h2>
<p>As mentioned previously, the installation capabilities on a Unix system mean
adding an <em>install</em> target to the Makefile.</p>
<p>Since we do not have root access on the CAE machines, we can try it by
installing to an alternate location where we have write permissions.:</p>
<pre class="literal-block">
mkdir ~/cxx-example-install
make DESTDIR=~/cxx-example-install install
</pre>
<p>Try changing the value of the <em>CMAKE_INSTALL_PREFIX</em> with ccmake and examine the
result by issueing the install command again.</p>
</div>
<div class="section" id="exercise-add-a-new-class-to-your-build-system">
<h2>Exercise: Add a new class to your build system</h2>
<p>Read through the contents of the top-level CMakeLists.txt in the example and see
if you can understand how it works.  The <a class="reference external" href="http://cmake.org/cmake/help/cmake-2-8-docs.html">CMake documentation</a> may also be
helpful.</p>
<p>You receive an email from your colleage in Veterinary Science.</p>
<blockquote>
<p>Hi SuperLeetHaxor,</p>
<p>I made a fantastic discovery.  We determined how to make our rabbits into
cannibals!  We call them CanniBunnies.</p>
<p>It took some genetic tweaking, but that didn't quite do it.  We found that we
had to have them watch continuous TV broadcasts of politicians running up the
national debt at the expense of their countrymen and future generations.  This
seemed to do the trick.  There were a couple of side effects.  Every time they
open their mouth, their nose somehow grows slightly.  And their libido is
increased, even for rabbits.</p>
<p>Anyway, I am interested in what their population dynamics would be, so I made
a new class to model them with your program.  You can get the new code by
applying the attached patch:</p>
<blockquote>
patch -p4 &lt; Make-a-Cannibal-class.patch</blockquote>
<p>However, I didn't know how to add the new class to your build system.  Can you
figure it out for me?  I would be grateful, and I promise not to eat you in
the future.</p>
<p>Regards,
CreepyRabbitGuy, PhD</p>
</blockquote>
<p>The file <tt class="docutils literal"><span class="pre">Make-a-Cannibal-class.patch</span></tt> can be found in the example source
directory.</p>
</div>
</div>
<div class="section" id="python-example-with-distutils">
<h1>Python example with distutils</h1>
<p>Python many different modules that provide build systems, but the build system
is typically implemented in a file at the root of the source tree called
<em>setup.py</em>.  Examples of different build systems include <a class="reference external" href="http://docs.python.org/distutils/introduction.html">distutils</a>,
<a class="reference external" href="http://distutils2.notmyidea.org/">distutils2</a>, <a class="reference external" href="http://pypi.python.org/pypi/setuptools">setuptools</a>, and <a class="reference external" href="http://pypi.python.org/pypi/distribute">Distribute</a>.  We will look at an examples that
uses <a class="reference external" href="http://docs.python.org/distutils/introduction.html">distutils</a> since it is the oldest and most widely available module.  Note,
though, that more modern system like <a class="reference external" href="http://pypi.python.org/pypi/distribute">Distribute</a> may be better.</p>
<p>Some Python build systems include support to build C extensions, but the example
we will look at today only 'compiles' and installs pure Python modules.</p>
<p>Get the example code:</p>
<pre class="literal-block">
svn checkout http://hacker-within.googlecode.com/svn/trunk/hacker-poke
</pre>
<p>Try installing:</p>
<pre class="literal-block">
cd hacker-poke
python setup.py --help
python setup.py install --help
mkdir ~/hacker-poke-install
python setup.py install --root=~/hacker-poke-install
</pre>
</div>
</div>
</body>
</html>
